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Abstract. We define a class of functions termed "Learnable in the Limit", 
based on the Machine Learning paradigm of "Identification in the Limit". A 
function is Learnable in the Limit if it defines a property P p of a recursively 
*— I ■ enumerable class A of recursively enumerable data sequences S € A, such 

that each data sequence S is generated by a total recursive function S that 
enumerates S. Let the index s represent the data sequence S. The property 
Pp(s) = x is computed by a partial recursive function 4>p{s,t) such that there 
00 1 exists a u where <t> p {s,u) = x and for all t > u, (f> p (s,t) = x if it converges. 

Since the index s is known, this is not an identification problem - instead it is 
learning a common property of the sequences in A. We give a Normal Form 
Theorem for properties that are Learnable in the Limit, similar to Kleene's 
Normal Form Theorem. We also give some examples of sets that are Learnable 
^ ) ' in the Limit, and derive some properties of Canonical and Complexity Bound 

• ' Enumerations of classes of total functions, and show that no full enumeration 

^ , of all indices of Turing machines TMi that compute a given total function f(x) 

can be Learnable in the Limit. 
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There has been a considerable amount of work on hypercomputation - functions 
that can be formally defined but are beyond the capability of any Turing machine [3] . 
^T") ■ One well-known example of a hypercomputation is that of Incompressible Numbers 

from Kolmogorov Complexity }3}. Kolmogorov Complexity is the function K(x) = y, 
where Turing machine TM y is the machine with the shortest index y that, starting 
with an empty tape, halts with x on its tape. The Incompressible Numbers are 
■ numbers where, essentially, K{x) = \x\ + c. This happens when there is no shorter 

way of generating these numbers except by storing a copy of the number x in the 
state table of the Turing machine, which is just the size of x plus a constant c. 
These values are known to be immune - no infinite recursively enumerable set is a 
subset of the Incompressible Numbers. 

Sets such as K have a conceptual genesis in Machine Learning. Kolmogorov 
based his complexity theory on work by Ray Solomonoff 6 [7 . Solomonoff, and 
later Mark Gold[2], developed some initial results on a type of Machine Learning 
called Identification in the Limit. Identification in the Limit is a learning task where 
the learner is presented with a sequence of elements from a recursively enumerable 
set and is given the task to determine an index % for the function cf>{ whose domain 
is the set. 

In this paper, we will consider a more general problem. We will start with the 
data set and its identification (pi and try to discover properties of this data. This is 
a learning task, but not an identification task - we know its identity already, but we 
want to discover interesting characteristics of the data. In the case of Kolmogorov 
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complexity, the data set is a single value x and the property is the shortest function 
that computes that value. 

The properties that we are looking for are learned in the limit - we make a guess, 
or find an initial result and update our hypothesis as we go along. This yields a 
nice normal form to characterize this class of learning tasks, similar to the Kleene 
Normal Form Theorem. We will also explore some learning tasks that are Learnable 
in the Limit and some that are not. 

2. Definitions 
Using the notion of Rogers [5] we define the Tau function as 

r ( x , y) = -jip 2 + 2x v + y 2 + % x + v) 

Where 

T 1 (a;) = x 
T 2 (x,y) = T(x,y) 

T k+1 (xi,...,X k+ l) = T(T k (x!,...,X k ),X k+ l) 

and Trf(x) is the inverse function for the i th element of r k . 

Since we are dealing with functions that are given as an input an initial segment 
of an infinite sequence, we assume that a function can be given a sequence of 
variables (S(0), S(n)} as an encoded input x using the tau function 

S = (S(0), S(n)) = r"+ 2 (n, S(0), S(n)) = x 

We use the notation TMi for the Turing machine with state table i. TM{(0) is 
the computation on an empty tape. TMi(x) is the computation on input x. An 
equivalent notation is fa, (f>i(0) and 4>i{x), which is the partial recursive function 
equivalent to the Turing machine TMi. The time complexity on input x is given 
as $>i(x), which is the number of steps that TMi(x) takes until it converges. A 
function with multiple inputs, for example <fii(x,y,z), is computed by a Turing 
Machine TMi(x,y,z) = TM i (r 3 (x,y,z)). 

If a set A is enumerable by a recursive function (pi, then <fii is a total function 
whose range is exactly A. 

We begin with a very general definition of Learning in the Limit for a single data 
sequence, where this input data sequence can be anything. 

Definition 2.1. A property P p is Learnable in the Limit on an Sequence S, 

where S is an infinite sequence of inputs S = ((5(0), S(n), ...), and 
Pp = 9 is a partial recursive function whose inputs are initial segments of S, 
where there exists a u such that g ((5(O), S(u))) — x and for all t > u, we have 
4> q ({S(0), S(t))) — x if it converges. If this happens, we say P p Converges on 
S in the Limit, or Z-converges to x. 

In this case, P P (S) = x, that is, P p learns x in the limit as a property of the 
sequence S. If there is no final value x, P p changes its mind an infinite num- 
ber of times and therefore P p i-diverges. Note that P p can be partial, so that 
<pq((S(0), S(n))) can fail to halt for some arbitrary (even infinite) number of 
values n. We use the notation P p (S) = x to indicate that P P {S) is Learnable in 
the Limit on S as x. 
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So the learning procedure must be a partial recursive function. But since the 
input data sequence can be anything at all, the problem is hard to characterize. 
For the sake of tractability, we will assume that the sequences are recursively enu- 
merable: the sequence S is the output of a total recursive function cf> s . 

Lemma 2.2. If S is recursively enumerable by a total function 4> s (n) = S{n) then 
P P {S) is a partial recursive function <j> p {s,t) where, if P p (S) 1- converges to x then 
there exists a u such that (p p (s,u) — x and for all t > u if <fr p (s,t) converges, then 
(f> p (s,t) = x. 

Proof. Trivial. <f> p (s,n) — <p q {<p r {s,n)) where 

<}> r {s,n) =T n+2 (n,cf> s (0),-,Mn)) 
since the S(n) values are generated by the function <f> 3 . □ 

If S is the output of a total recursive function (f> s then we can replace the sequence 
in the input of the property P P (S) with the index of the total recursive function 
that enumerates it: P p (s). 

Definition 2.3. If S is recursively enumerable by a total function <f> s (n) = S(n), 
and P p (S) is Learnable in the Limit for that Sequence S, then we use the notation 
P p (s) — x where P p is a partial recursive function 

4>p{s,y) = <t>q((<t>s(0), ...,<f> s (y))) 

where there exists a u such that <f> p (s, u) — x and for all t > u if <f> p (s, t) converges, 
then <fi p (s,t) — x. We say that P p (s)is computed by <j) p (s,t). 

The properties that are Learnable in the Limit can apply to a whole class of 
sequences that have the same learnable property P. If the class of sequences is 
a recursively enumerable class of recursively enumerable sequences, then we can 
replace the sequence in P(S) with P(s), where for all n, <p s {n) = S(n). 

Since we know the index s of a generating function for the data set, we are not 
identifying the set itself, but instead we are learning properties of the set. That is 
why we use the term "Learning in the Limit" . 

Definition 2.4. Assume that A is an infinite recursively enumerable class of re- 
cursively enumerable sequences S, where the range of total function (f> a are values 
s such that if S s G A then for all n, <j> s (ri) = S s (n). Then for all <p a (n) = s, 
P(s) = y for some y or P(s) Z-diverges. Let property P p be computed by 4> p for 
every sequence in A. This property P p (x) over A is Learnable in the Limit. The 
property P p is Z-Total if P p never ^-diverges on any input sequence in A. Otherwise 
P p is /-Partial. 

Note that the set of all shortest algorithmic descriptions of Kolmogorov com- 
plexity is expressed by a such property P K . P K takes the input x and generates a 
sequence (x, x, x, ...) at time n of size n. It is trivial to show that for every value 
x, there is a Turing machine TM Z defined uniformly on x, that just encodes the 
digits of x in its state table. Thus, z = \x\ + c. Therefore any shortest encoding 
of x must be shorter than z. So for each initial segment of the sequence of size n, 
P(x) runs TMo(O) through TM z (0) each for n steps and outputs y if there exists 
a value y < z where TM y (0) halts in less than n steps and outputs x. Otherwise it 
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outputs z. Then Pk /-converges to z only if x is incompressible. This property is 
/-total. 

The sequences (x,x,x, ...} used in this formulation of Kolmogorov complexity 
are not just a special case. It is trivial to show that given such a class A, for every 
property P p {s), S s S A, there is an equivalent class Az and an equivalent property 
Pp>(s'), S s i £ Az where the sequence S s > is simply (s, s, s, ...). We will define a 
special class composed of these sequences. 

Definition 2.5. Az is an infinite recursively enumerable class of recursively enu- 
merable sequences S where for all x, 4>z{%) = y, and for all n, 4> y {n) = x. 
Therefore S x G Az is the sequence (x,x,x, ...}. The inverse function is simply 
0z 1 (2/)=^(O) = a ; . 

Lemma 2.6. Assume that P p (x) over A is Learnable in the Limit. Then there is 
an equivalent property P p >{y) over Az where for all x there exists a y such that 
P p (x)=P p ,(y). 

Proof. Trivial. Let P p (x) over A be computed by 

^(x,t) = 0,((^ x (O),...,0 a (t))) 
Let </>2 1 (j/) = x. Define P p >{y) over Az to be computed by 

^(y,t) = g ,(<<^(o),...,<^(^ 

□ 

From this point on, we will assume that all properties P p are over Az- 
3. The Normal Form Theorem 

The notation P p {s) for properties Learnable in the Limit looks similar to the 
standard definition of a partial recursive function 4> p (s). Actually the only difference 
is the terminating condition. This allows us to construct a Normal Form Theorem 
that is similar to Kleene's Normal Form theorem except that instead of using the 
mu function /x(x) to find the first element where a Boolean predicate is true, we 
use a new function X(x) to find the last of a finite number of guesses. 

Definition 3.1. The function n(x)[...x...} is the least integer x such that the ex- 
pression ...x... is true when u x" is interpreted as the integer x, if ...x... is true at 
least one point [5]. 

Definition 3.2. The function X(x)[...x...] is the largest integer x such that the 
expression ...x... is true when "cc" is interpreted as the integer x, if ...x... is true at 
a finite number of points. 

Theorem 3.3. Kleene's Normal Form Theorem: There exists a primitive 
recursive function U and a primitive recursive predicate T(e, x, y) such that every 
function f(x) is effectively computable iff f(x) = 4> e {x) = U(fj,yT(e,x,y)). 

The Boolean function T(e, x, y) encodes in the variable y the computation history 
of Turing machine TM e on input x. The predicate T(e,x,y) returns true if y is a 
halting sequence. The function U recovers the output from the computation history 
y. We shall assume that if T(e, x, y) is true then for all z > y where z is y with one 
or more copies of the last tape configuration of y appended to y, then T(e, x, z) is 
true also. 
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Theorem 3.4. Learning in the Limit Normal Form: There exists a primitive 
recursive function U and a primitive recursive predicate T'(e,x,y) such that every 
property P p (x) is Learnable in the Limit iff P p (x) — U(\yT'(p,x,y)). 

Proof. By the definition of P p (x), there is a (j) p {x,n) that computes P p . By defi- 
nition, (f> p (x,n) = (f> p ((x,n)) = TM p (t(x,ti)). By Kleene's Normal Form Theorem 
there is a U and T such that 

<f> p (x, n) = U {fJ,yT{p, t(x, n),y)) 

Define T' from T as follows (for each p and x): 

T'(p,T(x,n),y) is true iff T(p,r(x,n),y) is true and for all to < n and z < y 
where T(p,T(x,m), z) is true, then U(y) ^ U(z). 

Note that this condition does not use the (i function. 

Assuming T(p,x,y) and U(x) are primitive recursive, T'(p,T(x,n),y) is also. 
If the following two conditions are true for some time n: 

• Let {(ii, yi), (i n , y n ), ...} be the set of all pairs such that for each n, 
T(p,r(x,i n ),y n ) is true. 

• For all m > n if T(p, t(x, i m ), y m ) is true then U(y n ) = U(y m ) 
then the following must be true: 

If yt is the largest value in the set {y , ■■■yn}, then T(p, t(x, yi), z) is true, where 
z > yi and z is a configuration with one or more copies of the last tape configuration 
in yi appended to z. 

So U(z) is the last value where TM p changes its mind on the input sequence. 
Therefore, the last time T'(p, x, y) is true is exactly this value y = z. 

SoP p (x) = U(\yT'(p,x,y)). □ 

Since this is similar to Kleene's Normal Form Theorem for the general recursive 
functions except that the [i in the normal form is replaced by A, we can tighten 
the Normal Form theorem to be a predicate T(e, x, y) that is true for at most one 
value y. 

Corollary 3.5. Revised Normal Form Theorem for Effectively Computable 
Functions: There exists a primitive recursive function U and a primitive recursive 
predicate T"(e, x, y), that is true for at most one point y for each pair e and x, such 
that every function f(x) is effectively computable iff 

f(x) = <p e (x) = U(fiyT"(e,x,y)) = U(XyT"(e,x,y)) 

Proof. Define T"(e, x, y) to be true iff T(e, x, y) is true and for all z < y, T(e, x, z) 
is false. □ 

This means that a property that is Learnable in the Limit is effectively com- 
putable if and only if there is an associated function that makes a single guess and 
never changes its mind. 

4. Some Properties Learnable in the Limit 

Here are a number of properties that are Learnable in the Limit but not effec- 
tively computable. These are all pretty obvious, so I shall just state them. 

Note that Learnable in the Limit properties can be /-partial functions, where 
for P p {x) there are either an infinite number of guesses or no guess at all. Some 
of the examples here are Z-total P p (x) functions. In all of these cases, the class of 
sequences is Az: 
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• The Kolmorogorov set K(x) — Pk{x) = y. The description was given in 
the Definitions as an example. Pjc is /-total. 

• Kolmogorov incompressible numbers: Pi(n) = x, where x is incompressible. 
Use Pr(x) and discard any values y that are compressible. When that 
happens, shift the guesses for Pi(n) down. At each value n there comes a 
time when all of the previous compressible values are found, and Pi{n) is 
never shifted. Pi is /-total. 

• The set of all partial recursive functions TM X where Pp(x) = (x,y) only 
if TM X is partial and y is the smallest value where TM x (y) diverges. The 
value Pp(x,y) is computed as follows: for all z < y run TM x (z) for y steps 
at most. If there is a value z where TM x {z) has not converged in y steps 
or less, output (x,z). If they all converge, output (x,y). If TM X is total, 
then if $ x (y) < y for all but a finite y, then Pp(x,y) — (x,y) for all but 
a finite number of cases and does not /-converge. Otherwise, the output 
of Pp{x,y) changes infinitely often as the smaller inputs converge. In this 
case, Pp is /-partial. 

• The set of all partial recursive functions TM X where for every n there is 
a unique x such that Pq(ti) = x. Use the same trick as above for the 
/-total property of Kolmogorov incompressible numbers. In this case, we 
must keep track of both the function TM a and the value Pp(a) — (a,b) 
at which it diverges. Ordering the (a, b) pairs, Pq(ji) outputs the value c, 
where (c, d) is the n th smallest pair. This takes /-partial Pp and turns it 
into /-total Pq. 

• The set of all partial functions with finite domain. Instead of keeping track 
of (a, b) where TM a (b) diverges, keep track of (a, (bo, b n )) where TM a (bi) 
converges, for each i < n. 

• The minimum index for finite sets - similar to the Kolmorogorov set. 

• The set of all functions TMi with a single element in its domain. 

• Given an enumeration of indexes of Polynomial functions A and enumer- 
ation of indexes of NP functions B, the Boolean property over pairs 
where P eq ((i,j)) — 1 iff i £ A, j G B and <fo = <f>j. Otherwise P eq ((i,j)) = 
0. If i A or j £ B, then P eq ((i,j)) diverges. 

• The property P exp (i) = (i,e) where </>, has polynomial complexity with 
exponent e. If <pi is not polynomial, then P exp {i) /-diverges. 

The Polynomial - NP property can be generalized to any pairs of recursively 
enumerable classes of total functions. We will present two types of generalizations. 
Let the class of total functions A be enumerated by a total function <j) a whose range 
is indexes of total functions in the set A, and similarly for B and A theorem by 
Blum and Blum on the extrapolation of total recursive functions uses the concept of 
an h-easy function. They show that each recursively enumerable class of functions 
is bounded, up to a finite number of exceptions, by the computational complexity 
of a total function /i.Q] 

Definition 4.1. Let h be a total recursive function. A partial recursive function 
(pi is h-easy if < h(x) for all but a finite number of integers x. 

This definition differs slightly from Blum and Blum's in that <pi can diverge in a 
finite number of cases. 
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Theorem 4.2. For any two total recursive functions g and h, P eq is Learnable in 
the Limit, where if fa is g-easy and fa is h-easy then P eq ((i,j)) = 1 if fa = 4>j and 
P eq ((i,j)) = if fa- ^ <f)j and P eq ((i,j)) diverges if either fa- is not g-easy or fa: is 
not h-easy. 

Proof. The function <f) eq ((i,j),t) is computed as follows. For all x < t, run fa(x) 
for at most g(x) steps and run faj(x) for at most h{x) steps. If y is the largest 
value where <&i(y) > g(y) or $>j(y) > h(y) or both, then output y + 1 unless y + 1 
was output before at some time. Otherwise, output if there is a case z where 
4>i{z) 7^ faj(z). If none are found, then output 1. 

The output y + 1 is a guess that fa is g-easy and fa is h-easy where all exceptions 
are less than or equal to y. From that point on, <p eq guesses 1 (the two are equal) 
until an exception is found. □ 

So the Boolean test of equality for h-easy classes of total recursive functions is 
Learnable in the Limit. 

Note, though, that neither the class of Polynomial functions nor the class of 
NP functions can be defined in this way, because the h-easy function will grow to 
include all polynomial exponents. This allows for functions whose computation time 
is not within any exponent. We can, though, enumerate indexes of the polynomial 
functions of the NP functions, and use them instead of g and h. 

Theorem 4.3. For any two total recursively enumerable sets A and B of total 
recursive functions, P eq is Learnable in the Limit, where if fa 6 A and fa: £ B 
then P eq ((i,j)) = 1 if fa = fa and P eq ({i,j)) = if fa ^ fa, infinitely often, and 
P eq ((i,j)) diverges if either fa £ A or fa £ B. 

Proof. In this case, fa q ((i,j),t) checks that fa € A and fa G B before running the 
test for equality. □ 

Note that these two theorems do not work for sets of partial recursive functions, 
because we can get stuck on a case where fa{x) converges and fa(x) diverges. The 
procedure does not identify this case, so it will return equality even though one 
function diverged. 

It is also possible to Learn in the Limit the ratio of equal to unequal values in 
the two classes, if the ratio is a rational number bounded in the limit. 

Definition 4.4. Given two total functions F and G, the Error of G on F (or 

F on G) is the value Err(x, F, G) = \{y < x\F(y) ^ G(y)}\ /x. This is a rational 
number in the range to 1. 

Definition 4.5. The Error of G on F Converges to v if there exists an error 
bound e(x) = a/b such that for each x, the value a/b is a rational number in the 
range to 1, where e(x) is subject to the following two conditions: 

• For all u and v, if u < v then e(u) > e(v). 

• For all rational numbers c/d in the range to 1 where c/al ^ there exists 
a t where c/d > e(t). 

Then there exists an m such that for all n > to if Err(n, F,G) = w then 
\v — w\ < e(n). 

Theorem 4.6. For any two recursively enumerable classes of total recursive func- 
tions, A and B, enumerable by fa, and fa, and an error bound e(x), 

Perr((i,j)) = (i,j,X,y) 
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is Learnable in the Limit, where for some n and m, 4> a {n) = i and fa(m) = j and 
the error of fa on fa converges to a rational number x/y. 

Proof. The function 4> er r'({hj),t) is computed as follows. Run fa(x) — fa(x) for 
each x < t. Compute Err(t,fa,faj) = w. Find the rational number a/b with the 
smallest denominator b such that \w — a/b\ < e(n). 

The assumption is that the error of fa on fa: converges to a rational number x/y. 
So there is an m such that for all n > m if Err(n, F,G) = w then \w — x/y\ < e(n). 

If e/f is a rational number where e/f ^ x/y then there exists a rational c/d 
where \e/f — x/y\ = c/d ^ 0. So there exists a t where c/d > e{t). For all 
s > t, \w — e/f\ > e(n) so e/f is not chosen after time t. This is true for all 
rational values e/f where / < y so all rationals in the range to 1 are rejected 
in favor of x/y at some time r. Therefore 4'err'{(i,j},' r ) — x/y and for all s > r, 

farr'({i,j),s) = farr'({i,j),r). □ 

Note that, although this theorem does not apply to the reals in general, any real 
number that can be computed as the output of a Turing machine starting with a 
blank tape (such as e or tt) can be added to this function by using the index of the 
associated Turing Machine as a possible output. 

Given a Learning in the Limit Problem P p where the output is a class of total 
functions fa, we can generate a Canonical listing, where every function appears 
only once. 

Definition 4.7. A Canonical Enumeration of total functions is a Learning in 
the Limit Problem P p where every function is unique. That is, for any x and y, if 
P p (x) = i and P p (y) = j then there is a z such that fa(z) ^ <fij{z)- 

Theorem 4.8. Assume a class of total functions is Learnable In the Limit by P , 
in the sense that for each x, if P(x) — y, then <p y is a total recursive function. 
Then there is a l-total P' where an index of each function is given exactly once. 

Proof. P' is derived from P where we only add in a function if it differs with each 
of the previous functions by at least one input. If P changes its mind, we have to 
recompute the differences. Eventually, each output for P' ^-converges. □ 

Definition 4.9. A Complexity Bound Enumeration of a canonical enumera- 
tion of total functions is an enumeration of all values i where there is a fa in the 
canonical enumeration such that 

• For all x, fa(x) = <pj{x) 

• For all x, &i(x) < <F, (x) 

A Complexity Bound Enumeration finds all of the algorithms that are faster 
than the one in the Canonical Enumeration. 

Theorem 4.10. If a class of total functions is Canonically Enumerable, then the 
complexity bound enumeration is Learnable in the Limit. 

Proof. Given the canonical listing, diagonalize each canonical index i over the enu- 
meration of all Turing machines TMj, discarding any that differ (fa(x) ^ </>j(a;)) 
or take too much time ($i{x) > <&j(x)). □ 
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5. Some Properties That Are Not Learnable in the Limit 

We end with a couple of examples of properties that are not Learnable in the 
Limit. 

Definition 5.1. A Complete Enumeration of a class of functions A is an /-total 
property P p , Learnable in the Limit, such that, for all TMi € A, every function 
TMj = TMi has a value x where P p (x) = j. 

Theorem 5.2. The Complete Enumeration of all Total Recursive Functions is not 
Learnable in the Limit by any 1- Total P p . 

Proof. Assume the opposite: the complete enumeration of all total recursive func- 
tions is Learnable in the Limit by a /-Total P p . 

Since <j) p is /-total then for each x there exists a u such that for all t > u if <fi p (x, t) 
converges, then cf) p (x,t) — <f) p (x,u) — y and 4> y is a total recursive function. Also, 
every index j of a total function TMj is an output of P p {v) for some v. 

Define a family of functions f(i) where for each i, (f>f^((x,u)) is constructed 
from (p p (x,t) as follows: begin by running (j) p (x,t) for all t < (u + i) for (u + i) 
steps at most. If no such 4>p{ x i t) converges in time (u + i) or less, then dovetail the 
computations of (j) p (x, i) for all t, and find the first v where 4> p (x, v) converges. If w 
is the largest value where <fi p (x, w) converges in this computation and 4> p (x, w) = y, 
then run (j) y ({x, u)). If 4> y ({x, u)) converges, where <f> v {{x, u)) — z then output z + 1. 

By the assumption, for all x, P p (x) /-converges to the index of a total recursive 
function y, although the initial guesses may be of partial recursive functions. If 
4>f{i){{ x i u )) uses one of these guesses, it will never halt, at least on a finite number 
of initial values of u. But there will come a time s where for all t > s the function 
<f)f^{{x, u)) converges to a value for every u and is therefore total. 

Choose j, f(t) — j where 4>j is total. Since P p is a complete enumeration, there 
will be an index y such that P p (y) = j in the limit, so after a finite value u, for 
all t > u <j) p ((y,t)) = j if it converges. Let <f)j((y,t)) = z. This value z exists, 
since is total. But by the construction of 4>f(t) — 4>j given above, <f>f( t )((y, s )) = 
0i((z/i s )) — < Pj((yi s )) + 1 a contradiction. So the Complete Enumeration of all 
Total Recursive Functions is not Learnable in the Limit by a /-total P p . □ 

We can strengthen this result by showing that no Complete Enumeration of even 
a single total recursive function is Learnable in the Limit by any P pi even if it is 
/-partial. 

Theorem 5.3. Given any total recursive function TMi, the Complete Enumeration 
of all Total Functions TMj equal to TMi is not Learnable in the Limit by any P p . 

Proof. Assume the opposite: for an arbitrary TMj, there is a P p such that for all 
j, TMi — TMj iff there is an x such that P p (x) = j. Let P p (x) be computed by 
(f> p (x,t). 

Given i and <f) p , define TM n (y) as follows. Run all (j) p (x,t) computations for 
x < y and t < y for up to & p (x, t) < y. If there is no case where (j) p {x, t) — n then 
TM n {y)=TM t {y)- 

Otherwise, enumerate all cases (x,t) where (f) p (x,t) = n for x < y and t < y in 
time & p (x, t) < y. Let y — T(m, q) and select the m th case (z, s) where <j> p (z, s) = n. 
This ensures that each such case gets chosen an infinite number of times during the 
computations of all inputs y to TM n (y). Dovetail the computations of all 4> p (z, r) 



10 



ANTONY VAN DER MUDE 



for r > s until a value v is found where 4> p (z,v) — m and m ^ n. If none is found, 
then TM n (y) diverges. Otherwise, TM n (y) = TA'h{y). 

If P p (y) never equals n for all y, then TM n = TMi. Then P p cannot be a 
Complete Enumeration, since it missed TM n . 

If P p (y) — n for some y then there is a value s where <f> p (y, s) = n. At this point 
4> p (y, t) either diverges or <j> p (y, t) = n for all t > s. Let TM n (v) be a value v where 
this value 4> p (y, s) = n is the case selected in the computation of TM n (v). By the 
construction of TM n , TM n (v) diverges, and is therefore not a total function. 

Since this construction is true for any % and <f> p , the Complete Enumeration of all 
Total Functions TMj equal to TMi is not Learnable in the Limit by any P p . □ 

6. Conclusions 

The class of properties that are Learnable in the Limit are an interesting exten- 
sion of the effectively computable functions. Although they are not computable in a 
finite time, they model our everyday notion of what learning and generalization are. 
The restricted subset of problems termed "Identification in the Limit" have been 
extensively covered in the Machine Learning and Computational Learning Theory 
fields. But the formal properties of the class itself, outside of its use as a model for 
learning, is itself interesting. This paper serves as a start for the exploration of this 
class. 

It is obvious that the substitution of fi(x) for X(x) in the Normal Form Theorem 
can be further extended to classes where the computing function changes its mind 
an infinite number of times, and there is some predicate that defines a sort of limit- 
ing condition of that sequence. This would make it possible to extend Theorem l4.6l 
to the reals. This extension would of course be another superset, with character- 
istics all its own. How these levels of computation correspond to the Arithmetical 
Hierarchy is an open question. 
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